in_cali for fire points restricted
to California’s borders.They record fire detection data several times a day (around once every 2 hours), but there can be a little delay in making the data available (this doesn’t affect the observation time recorded).
Each observation represents a fire detection point recorded on a satellite for a specific time of a date at one location.
The variables provided are:
In particular, FRP is the rate of emitted radiative energy by the fire at the time of the observation, expressed in units of power (i.e. Watts). It can be used to gauge fire intensity.
There is correspondence between method of detection and satellites:
From the bar chart, we can see that the following ecosystems are the most prominent during the Californian fire:
Refer to the dataset smoke to see details.
Each observation represents a unique smoke polygon of a specific level recorded at a certain time by one satellite.
The variables included in the smoke dataset are:
We use HDBSCAN to cluster fire detection points for each day.
To find the best-performing clusters for each day, we built
HDBSCAN using minPts ranging from 5-100 with step size = 5.
Model performance is measured using CDbw. It considers
multiple representative points per cluster, but it means that we have to
introduce a parameter for defining the number of points. CDbw index (a
type of relative validity index) is calculated based on the following
metrics:
The higher the CDbw index, the better the clustering compared to the others.
The following table was created using the dataframe
cluster_info.
| Cluster | Number of Observations | Cluster Area | Cluster Density | Average FRP | FRP Variance |
|---|---|---|---|---|---|
| 2018-11-08_1 | 13 | 1.423410e+07 [m^2] | 0 [1/m^2] | 2.0753 | 0.5137 |
| 2018-11-08_10 | 10 | 3.176393e+08 [m^2] | 0 [1/m^2] | 4.2535 | 2.6519 |
| 2018-11-08_11 | 7 | 5.394937e+07 [m^2] | 0 [1/m^2] | 1.1526 | 0.2596 |
| 2018-11-08_12 | 6 | 8.113654e+05 [m^2] | 0 [1/m^2] | 9.1422 | 61.9808 |
| 2018-11-08_13 | 7 | 3.657163e+06 [m^2] | 0 [1/m^2] | 7.3448 | 62.7275 |
| 2018-11-08_14 | 9 | 9.207893e+05 [m^2] | 0 [1/m^2] | 24.1770 | 403.1557 |
| 2018-11-08_15 | 31 | 1.501041e+08 [m^2] | 0 [1/m^2] | 14.9712 | 106.6972 |
| 2018-11-08_16 | 8 | 1.365842e+08 [m^2] | 0 [1/m^2] | 2.4090 | 0.5152 |
| 2018-11-08_17 | 15 | 1.850766e+08 [m^2] | 0 [1/m^2] | 2.3319 | 2.0031 |
| 2018-11-08_18 | 2286 | 2.511210e+08 [m^2] | 0 [1/m^2] | 134.3567 | 85643.1991 |
| 2018-11-08_2 | 17 | 5.667023e+05 [m^2] | 0 [1/m^2] | 31.8651 | 1984.3889 |
| 2018-11-08_3 | 20 | 1.167228e+05 [m^2] | 0 [1/m^2] | NaN | NA |
| 2018-11-08_4 | 7 | 3.398569e+06 [m^2] | 0 [1/m^2] | NaN | NA |
| 2018-11-08_5 | 50 | 4.815113e+06 [m^2] | 0 [1/m^2] | NaN | NA |
| 2018-11-08_6 | 11 | 3.591449e+05 [m^2] | 0 [1/m^2] | 3.7036 | 9.9555 |
| 2018-11-08_7 | 38 | 1.202311e+06 [m^2] | 0 [1/m^2] | 16.6623 | 312.9396 |
| 2018-11-08_8 | 38 | 1.925391e+06 [m^2] | 0 [1/m^2] | 20.3447 | 884.1141 |
| 2018-11-08_9 | 9 | 2.235926e+06 [m^2] | 0 [1/m^2] | 1.8465 | 1.1100 |
| 2018-11-09_1 | 10 | 2.049449e+06 [m^2] | 0 [1/m^2] | NaN | NA |
| 2018-11-09_2 | 1599 | 1.002942e+08 [m^2] | 0 [1/m^2] | 58.6863 | 13224.1577 |
| 2018-11-09_3 | 2074 | 1.776215e+08 [m^2] | 0 [1/m^2] | 104.9371 | 54726.3656 |
| 2018-11-09_4 | 97 | 1.406530e+07 [m^2] | 0 [1/m^2] | 5.8576 | 38.0230 |
| 2018-11-09_5 | 169 | 2.571885e+08 [m^2] | 0 [1/m^2] | 6.9627 | 49.2537 |
| 2018-11-09_6 | 4018 | 2.732335e+10 [m^2] | 0 [1/m^2] | 32.2106 | 5520.4299 |
| 2018-11-10_1 | 105 | 3.081300e+06 [m^2] | 0 [1/m^2] | 18.7275 | 1102.8667 |
| 2018-11-10_10 | 1603 | 3.844738e+08 [m^2] | 0 [1/m^2] | 20.9242 | 1750.0464 |
| 2018-11-10_2 | 95 | 5.500144e+06 [m^2] | 0 [1/m^2] | 3.3426 | 10.0919 |
| 2018-11-10_3 | 13 | 5.084765e+05 [m^2] | 0 [1/m^2] | 3.0239 | 6.8927 |
| 2018-11-10_4 | 99 | 3.800582e+06 [m^2] | 0 [1/m^2] | 7.5454 | 24.7879 |
| 2018-11-10_5 | 93 | 1.838807e+06 [m^2] | 0 [1/m^2] | 20.6890 | 334.9789 |
| 2018-11-10_6 | 88 | 3.369599e+06 [m^2] | 0 [1/m^2] | 15.3668 | 149.7147 |
| 2018-11-10_7 | 604 | 2.699461e+07 [m^2] | 0 [1/m^2] | 14.3358 | 407.3974 |
| 2018-11-10_8 | 166 | 9.642361e+06 [m^2] | 0 [1/m^2] | 4.7362 | 13.0285 |
| 2018-11-10_9 | 14 | 4.280154e+05 [m^2] | 0 [1/m^2] | 4.5693 | 11.1143 |
| 2018-11-11_1 | 1967 | 8.197540e+08 [m^2] | 0 [1/m^2] | 27.2822 | 8698.4072 |
| 2018-11-11_2 | 274 | 1.957522e+07 [m^2] | 0 [1/m^2] | 85.8931 | 71447.4052 |
| 2018-11-11_3 | 128 | 9.327657e+06 [m^2] | 0 [1/m^2] | 38.6417 | 1660.4962 |
| 2018-11-11_4 | 114 | 1.112865e+07 [m^2] | 0 [1/m^2] | 5.4604 | 24.3280 |
| 2018-11-11_5 | 14 | 6.121560e+05 [m^2] | 0 [1/m^2] | 3.9945 | 22.6090 |
| 2018-11-11_6 | 12 | 5.606970e+07 [m^2] | 0 [1/m^2] | 1.9680 | 2.8212 |
| 2018-11-12_1 | 80 | 1.806843e+07 [m^2] | 0 [1/m^2] | 5.0002 | 18.5542 |
| 2018-11-12_2 | 62 | 8.770464e+06 [m^2] | 0 [1/m^2] | 9.7789 | 97.4233 |
| 2018-11-12_3 | 113 | 8.374984e+06 [m^2] | 0 [1/m^2] | 3.7706 | 18.7874 |
| 2018-11-12_4 | 50 | 2.184732e+06 [m^2] | 0 [1/m^2] | 9.8772 | 191.4502 |
| 2018-11-12_5 | 2683 | 1.519259e+09 [m^2] | 0 [1/m^2] | 15.7495 | 600.7126 |
| 2018-11-13_1 | 99 | 2.005938e+09 [m^2] | 0 [1/m^2] | 8.5016 | 33.9493 |
| 2018-11-13_2 | 44 | 5.901150e+06 [m^2] | 0 [1/m^2] | 2.9030 | 4.4938 |
| 2018-11-13_3 | 34 | 1.348693e+06 [m^2] | 0 [1/m^2] | 4.8680 | 11.0120 |
| 2018-11-13_4 | 546 | 1.096001e+09 [m^2] | 0 [1/m^2] | 5.6562 | 23.5390 |
| 2018-11-14_1 | 128 | 1.235797e+06 [m^2] | 0 [1/m^2] | 13.3365 | 206.0780 |
| 2018-11-14_10 | 56 | 8.396809e+06 [m^2] | 0 [1/m^2] | 2.8089 | 6.2422 |
| 2018-11-14_2 | 174 | 1.100831e+07 [m^2] | 0 [1/m^2] | 8.8678 | 167.3056 |
| 2018-11-14_3 | 364 | 1.426946e+07 [m^2] | 0 [1/m^2] | 13.1817 | 205.8078 |
| 2018-11-14_4 | 128 | 6.347390e+06 [m^2] | 0 [1/m^2] | 3.5365 | 9.0063 |
| 2018-11-14_5 | 19 | 1.138324e+06 [m^2] | 0 [1/m^2] | 3.2985 | 2.5204 |
| 2018-11-14_6 | 14 | 3.621616e+05 [m^2] | 0 [1/m^2] | 5.1807 | 69.5815 |
| 2018-11-14_7 | 26 | 1.196365e+06 [m^2] | 0 [1/m^2] | 5.7311 | 32.5206 |
| 2018-11-14_8 | 56 | 2.809779e+08 [m^2] | 0 [1/m^2] | 17.1428 | 457.6125 |
| 2018-11-14_9 | 22 | 1.284046e+06 [m^2] | 0 [1/m^2] | 2.8535 | 4.9944 |
| 2018-11-15_1 | 270 | 1.769762e+08 [m^2] | 0 [1/m^2] | 7.5457 | 118.7346 |
| 2018-11-15_2 | 32 | 1.990718e+06 [m^2] | 0 [1/m^2] | 2.3298 | 1.9905 |
| 2018-11-15_3 | 196 | 2.581558e+07 [m^2] | 0 [1/m^2] | 13.2800 | 331.1545 |
| 2018-11-16_1 | 19 | 2.129944e+06 [m^2] | 0 [1/m^2] | 1.5167 | 0.7461 |
| 2018-11-16_2 | 62 | 3.057250e+06 [m^2] | 0 [1/m^2] | 13.6501 | 566.9059 |
| 2018-11-16_3 | 34 | 2.637650e+06 [m^2] | 0 [1/m^2] | 5.4419 | 50.2059 |
| 2018-11-16_4 | 14 | 3.714076e+05 [m^2] | 0 [1/m^2] | 6.0042 | 28.5806 |
| 2018-11-16_5 | 23 | 9.402680e+05 [m^2] | 0 [1/m^2] | 3.2401 | 8.6760 |
| 2018-11-16_6 | 276 | 9.934960e+06 [m^2] | 0 [1/m^2] | 9.5678 | 174.1921 |
| 2018-11-16_7 | 130 | 5.384844e+06 [m^2] | 0 [1/m^2] | 8.2047 | 60.4990 |
| 2018-11-16_8 | 173 | 3.733934e+06 [m^2] | 0 [1/m^2] | 8.7509 | 71.7737 |
For each PM2.5 observation, we will create three indicator variables based on whether it is (1) or is not (0) in the 1) light, 2) medium, and/or 3) heavy smoke plume of the same day.
We will also create a numerical variable indicating the distance to the closest cluster (as indicated by a single representative point) in meters.
Here’s the first five observations of the augmented AQS dataset
(variable aqs):
| PM25 | POC | Date | Latitude | Longitude | State | County | City | State.Code | County.Code | Site.Num | Site.Code | in_light | in_med | in_heavy | fire_dist | closest_cl | geometry |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 14.54167 | 3 | 2018-11-08 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 1 | 1 | 0 | 60855.64 [m] | 2018-11-08_15 | POINT (-121.7842 37.68753) |
| 74.20833 | 3 | 2018-11-09 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 1 | 1 | 1 | 237850.58 [m] | 2018-11-09_6 | POINT (-121.7842 37.68753) |
| 72.29167 | 3 | 2018-11-10 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 1 | 1 | 1 | 225617.57 [m] | 2018-11-10_7 | POINT (-121.7842 37.68753) |
| 61.16667 | 3 | 2018-11-11 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 1 | 1 | 0 | 240824.38 [m] | 2018-11-11_1 | POINT (-121.7842 37.68753) |
| 77.79167 | 3 | 2018-11-12 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 1 | 0 | 0 | 219628.85 [m] | 2018-11-12_5 | POINT (-121.7842 37.68753) |
| 50.58333 | 3 | 2018-11-13 | 37.68753 | -121.7842 | California | Alameda | Livermore | 6 | 1 | 7 | 06-001-0007 | 0 | 0 | 0 | 240868.56 [m] | 2018-11-13_4 | POINT (-121.7842 37.68753) |
```